Processor, network bandwidth, and memory performance have rapidly increased over the years. Storage however remains a bottleneck.
Magnetic storage has scaled in size with hard drives now offering terabytes worth of storage. Cost has also been reduced in terms price per megabyte of storage. However, performance improvements for this type of storage have only been incremental relative to performance improvements made to other hardware components of a computer system.
Solid state storage narrows the storage performance gap relative to the other hardware components by providing improved performance over magnetic storage. The tradeoff, however, is the price per megabyte of solid state storage is significantly higher relative to magnetic storage. Also, solid state storage capacities are magnitudes smaller than magnetic storage capacity.
In view of these tradeoffs, magnetic storage remains in wide spread usage and remains a primary option for storing large sized files. In fact, the transition to higher definition video content and more feature rich content enabled by the greater processing power and network bandwidth has ensured that magnetic storage will not be replaced by solid state storage or other storage options for some time. Game, application, and video content creators have little option but to continue using magnetic storage because of their larger storage capacities and lower cost of operation even if performance is not optimal.
Different storage mediums can be combined within a single computer system to leverage their different advantages and thereby improve overall storage performance. In a typical combined usage scenario, smaller but faster storage is used as a caching layer for larger but slower storage. The caching layer attempts to keep the most important (e.g., most frequently requested or most recently used) content in the faster storage medium so that it can be accessed more efficiently.
A file system controls when and how files are written to and read from the one or more computer system storage mediums. This allows the file system to also control how content is cached across the different storage mediums. Accordingly, the file system has a large affect on storage performance.
File systems rarely realize the full input/output (I/O) performance of the different storage mediums under their control. This is in large part due to the file system being optimized for general application. File systems attempt to provide the best performance for all possible storage usage scenarios. In other words, current file systems attempt to provide constant performance for storage (i.e., reading and writing) of small file sized content and large sized content, for storage of large sequential blocks as well as bursts of random blocks, and for storage of cached and uncached content as some examples. For these reasons, current file systems do not provide optimal performance for any particular usage scenario. As a result, hard drives are rarely used at their full performance capabilities. This in turn degrades overall computer system performance.
There is therefore a need to improve file system performance and, in so doing, improve storage performance within a computer system. There is a need to shift away from the paradigm of optimizing for the lowest common denominator or for providing balanced storage performance for all usage scenarios. Instead, there is a need to optimize storage performance for the primary usage scenario of the computer system, even if, such optimizations degrade performance for less common storage operations on the particular computer system. Thus, there is a need to improve how, where, and when data is written to and read from one or more different computer system storage mediums as well as how data is cached across the different storage mediums. To this end, there is a need to optimize the file system and hard drive usage in relation the real world storage of data on a computer system.